It has long been an axiom of computer science that “humans use inductive reasoning but computers use deductive reasoning.” Numerous computer science courses begin with a similar axiom and an explanation of the two types of reasoning as a way of coaching students to begin “thinking like a programmer,” with the expectation that a good command of deductive reasoning is necessary to perform any programming at all.
To provide background, deductive reasoning is a reasoning process that works from general axioms to more specific, logically certain conclusions. In deductive reasoning, a general rule is developed which holds over the entirety of a closed domain of discourse, and subsequent general rules are applied in order to narrow the range under consideration, allowing a conclusion to be reached reductively once only the conclusion is left as an option. A simple, commonly used example of deductive reasoning is “All men are mortal [a first premise], Socrates is a man [a second premise], therefore Socrates is mortal [the conclusion].” In such an example, the first premise requires all objects classified as “men” to have the attribute “mortal,” the second premise requires Socrates to be classified as a member of the set “men,” and as such the conclusion can be drawn that Socrates has the attribute “mortal” as a necessary aspect of his membership in the set “men.”
Three major reasoning processes may be applied as part of “deductive reasoning,” these being modus ponens, modus tollens, and syllogism. Modus ponens is the process described in the example described above, regarding Socrates; a first premise is provided, “If P, then Q,” and a second premise is provided, establishing “P.” Following the first premise and combining it with the second premise, it may thereby be established that, since P is true, and If P, then Q, then Q is true. Modus tollens is the contrapositive of this. “If P, then Q” is still established as a first initial premise, but as a second premise, it is then established that Q is not the case, the “negation of the consequent,” represented as “-Q.” Since, as per the first premise, Q must follow from P if P exists, and since, as per the second premise, Q is not present, then the natural consequence of P existing—Q also existing—has not been observed, and therefore P cannot exist. Syllogism is used to bridge premises. For example, “if P, then Q” can be established as a first initial premise, and “if Q, then R” can be established as a second initial premise, meaning that it can be reasoned that “if P, then R.” Since P existing would have to mean Q existing, and since Q existing would have to mean R existing, the two premises can be combined in this manner.
There are certain limitations to the process of deductive reasoning, which can limit its application. To start, a distinction must be drawn between the “validity” and the “soundness” of any argument made through deductive reasoning. An argument can be said to be “valid” when, assuming that the premises are true, the conclusion logically follows. “Valid” arguments can still be factually incorrect. For example, it may be said that “if P, then Q,” and “P” may be observed to exist, but “if P, then Q” may not be true as a universal rule, and may only be true some proportion of the time. Therefore, in a particular case, “if P, then Q; P; then Q” may be a valid argument, following logically from its premises, but may still be wrong, since it is based on faulty information or information that cannot be properly universalized. “Sound” arguments, on the other hand, are those which are valid and which have true premises. Computers, in particular, have trouble distinguishing between the two, and have little ability to evaluate soundness; for example, a premise that is true 99% of the time can cause significant amounts of difficulty.
Inductive reasoning works in an opposite manner to deductive reasoning, moving from specific observations to broader generalizations and theories. In inductive reasoning, a conclusion may be reached by generalizing or extrapolating from specific cases to general rules which govern all such cases.
Inductive reasoning can be best described by contrasting it with deductive reasoning. In contrast to deductive reasoning, which derives conclusion b from premise a only where b is the formal logical consequence of a, inductive reasoning is fuzzier, and can allow the inferring of b from a, where b does not follow necessarily from a. This means that conclusions drawn from inductive reasoning are based on some degree of guesswork as to what rule actually covers all potential cases, and such conclusions can often be wrong if the observed cases are not representative of the set of all potential cases. For example, if it can be reasoned that, if P is true, then consequently Q, R, and S are each true, deductive reasoning may, if P is established as true, likewise be able to establish Q, R, and S as true. An example of inductive reasoning, however, might have an observer establishing that, upon finding that Q, R, and S are simultaneously true (when Q, R, and S are all relatively independent and such a pattern is rare outside of P being true), P is likely to be true with some degree of probability.
A practical example of inductive reasoning might be as follows. If a number of swans are observed and all observed swans are white, a conclusion may be drawn (and associated with some degree of probability or certainty) that all swans are white. The conclusion is justifiable, but the truth of the conclusion is not guaranteed. (As some swans are black, it is actually wrong in this case.) As such, instead of being able to draw the conclusion of “All of the swans we have seen are white, therefore we know that all swans are white,” this conclusion must be associated with an epistemic expectation, i.e. “we expect that all swans are white (based on some level of certainty).”
A further reasoning method, abductive reasoning, is also sometimes necessary. Abductive reasoning is a process principally used in formulating hypotheses, and allows inferring a as an explanation of the cause of b, even without clear evidence establishing that “if a, then b” is a universal principle or premise of the sort that could be applied to deductive reasoning. Logically, abductive reasoning is equivalent to the fallacy of affirming the consequent, or “post hoc ergo proper hoc,” because of the possibility of multiple different explanations for b, meaning that a may play a comparatively smaller part than assumed or may even be associated with b as pure coincidence.
For example, given a set of observations O, a theory T, and a potential explanation E, for E to be abductively reasoned as an explanation for 0, then, overall, the system should satisfy two conditions. First, it must be established in some manner that O follows from E and T. Second, it must be established that E is consistent with T. Some “best explanation” can then be selected from multiple Es if multiple Es fit the requirements; often, this requires some sort of subjective evaluation of the simplicity of a given explanation, the prior probability of similar explanations, or the explanatory power of the explanation.
For example, if a white feather is found on the ground, it may be abduced that a swan was present and that the presence of the feather was due to the presence of the swan, because the presence of the swan (losing a feather) would account for the presence of the feather. This would provide a hypothesis explaining the observation. Plenty of alternative explanations may exist, however, which may be seen as less likely; for example, a person may have passed by with a feathered costume or feathered clothing (such as a feathered dress), or a person may have camped there with a down sleeping bag that lost some of its contents (from a rip or tear), or the feather may be from another bird entirely. It may also be conceivable that the feather blew to the location where it was observed from elsewhere. The potential explanations can be evaluated based on their simplicity; for example, if it is known that swans nest in the area and swans can be seen at a distance, the swan explanation may be the simplest, whereas if the feather is found within a ballroom, the swan explanation may require a further explanation of how the swan came to be in the ballroom, why the swan was not removed from the building before reaching the ballroom, and so forth, and a simpler explanation may instead be that the feather fell off of a costume. The potential explanations can also be evaluated based on the prior probability of such explanations being accurate; for example, if a large number of white feathers are observed to be dropped by swans in the area, any newly observed white feather may be likely to have come directly from a swan, even if the feather was not seen falling. However, if a large number of feathers actually come from campers with downy jackets or sleeping bags, and swans avoid the area because of the human presence, then it may be more probable that the feather was left by a camper. Finally, the potential explanations can be evaluated based on the explanatory power of the explanation to encompass other observations as well; for example, if a feather is found having glue on it, an explanation that the feather came from a costume may have more explanatory power (explaining the feather observation and the glue observation) even if swans are likely to nest in the area and people in costume are rarer. Likewise, if a feather is found next to a matted area of grass and a torn scrap of polyester fabric found on a rock, an explanation of the feather having been left by a camper may have much more explanatory power, as it may explain the presence of the matted grass (the camper pitched a tent there or unrolled a sleeping bag there) as well as the presence of the fabric together with the feather (the camper ripped their sleeping bag or jacket on the rock, and a feather came out).
As such, despite many such alternative explanations of a phenomenon, or a collection of phenomena that are observed, a single explanation (or a few explanations) will generally be abduced through this process, as a working hypothesis. Some possibilities can thus be disregarded. (This is suspected to have been an important evolutionary skill for humans, in that it may allow unfamiliar surroundings or unfamiliar situations to be navigated. However, attempts to implement such reasoning with computer systems have been fairly weak, finding a limited level of success in diagnostic expert systems and other such systems.)
As touched on briefly above, deductive reasoning has been much more successfully implemented on computers, and a variety of systems have been implemented for performing deductive analysis tasks on computers. Computers have been used to discover features and patterns in large data sets, to find correlated phrases in text, to analyze image data, and to prove logical theorems by deductive reasoning. This has allowed for large leaps in capability in certain areas, such as image recognition, as data has ballooned in availability. Computers can identify particular patterns in extensive image data sets (such as ImageNet's database of approximately 14 million images, many with manually-added identifiers and descriptors to improve reasoning) and identify features common to many or most of the images related to a certain topic. This allows for computers to greatly assist with the inductive reasoning processes that may be performed by a human operator or programmer. However, the ability to actually invent or understand ideas has remained uniquely human, or at least biological, to the point where even non-human animals are looked at as a more likely source of such origination than computers are. (To date, the United States has had at least one court case regarding the potential intellectual property rights of a non-human animal, but AI-derived IP has been a pure hypothetical.)
A simple example can be considered to demonstrate the existing limitations of computers with respect to automated reasoning. It would be possible, in principle, to present a machine learning program with examples of triangles. A computer program set up to test the relative lengths of the three sides may be able to validate an approximation of the Pythagorean Theorem within some error limit, if presented with enough examples of right triangles, obtuse triangles, and acute triangles. However, the empirical discovery of this approximation could never replace the exact formulation of the Pythagorean Theorem itself, which depends on a foundation of prior geometrical understanding, and which serves as a theoretical underpinning for many diverse areas of mathematics and other sciences that would be difficult to understand without it.
An ongoing trend in the field of artificial intelligence has been its evolution away from the oversimplified apparatus of rule-based systems toward systems not based on centralized control or processing. The large increase in the number of layers in newer convolutional neural network (CNN) systems has made possible the concept of “deep learning,” which has dramatically improved the ability to discriminate patterns in imagery, text, and other data by using these highly structured CNNs to identify patterns based on training of the CNN program.
However, detecting a pattern or feature in data is not the same as the understanding the pattern, interpreting the discovery in the context of other information or knowledge, or making theoretical predictions that synthesize broad areas of learning. Even despite advances in pattern detection, because of the limitations of these techniques, there are at present no machine learning systems that can compete with humans at all in areas like comprehending image or text data. The most effective systems, at present, are NLP systems used to try to gauge the sentiment or gist expressed by a text document from the statistical analysis of co-occurrences of words and phrases. Based on this analysis, a sentiment or summary (such as a two-line summary of a news article for a news ticker, or some similar result) may be produced with some accuracy, but only may be discoverable based on statistical data characterizing a large set of such news articles and consistency in the word choice and sentence structure of such articles. No significant amount of anything that could be called understanding of intentions, concepts, or purposes of statements in any media, from text to images to video, has been achieved.
For example, in video analysis, detection of movement is relatively simple, and identification of objects based on features often succeeds, but it is difficult for a computer to construct a narrative based on object recognition and a priori knowledge of objects in a scene. Even if the identity of each object and the positions of each object are “known” throughout the duration of the video, the computer has little to no ability to identify what is actually happening in the video without the same sorts of techniques used in image recognition in the first place. (For example, a machine learning-based reasoning system may be able to analyze a video clip of a person throwing a ball on the basis of still images in the video clip, and may be able to identify individual frames on the basis of their similarity to other images. For example, it may be able to identify that the video clip includes “man holding ball,” “man AND ball,” “man” (with the ball then being out of the frame), but may not be able to chain all of the aspects of the video clip into any sort of narrative. Similarly, natural language processing (NLP) can extract patterns in word groups in vast amounts of text data, but that is different from reading and comprehending the text. ML systems would readily fail the Turing Test, which requires computers to match human intelligence to a degree that makes them indistinguishable from humans.
Automated theorem-proving programs have been applied to automated reasoning problems with mixed success. By way of background, automated theorem-proving programs or algorithms, “ATPs” for short, are or use algorithms for proving mathematical theorems using computer programs. In such programs, knowledge is codified as “theorems,” assumptions and postulates are codified as “axioms,” and hypotheses are tested for truth or falseness by a process known as “paramodulation,” which consists of testing the consistency of the hypothesis with the prior knowledge contained in the stored axioms. Examples of success include hardware verification, proof verification, demonstration of the correct ness of computer programs, and model checking. Automated theorem proving algorithms have been applied in physics, geometry, and computer vision, among other areas. Automated theorem proving algorithms have been used as proof assistants in checking the correctness of mathematical proofs, and in some cases, have discovered proofs for unsolved conjectures.
However, in general, automated theorem proving programs need operator assistance and human control to operate successfully, due to factors such as a lack of flexibility. (For example, almost all ATP programs require a highly formatted input structure that is not only not necessarily compatible with most types of inputs but is not necessarily compatible with many types of data, as many types of data cannot be tested for consistency with the prior knowledge contained in the stored axioms by existing ATP software.) A basic strategy for using such systems is to apply inference rules to demonstrate that a conjecture is a logical conclusion of given propositions. The process may formalize logical systems in such a manner as to apply constructs from propositional calculus, and may use first-order logic structured as sentences containing assertions with variables. One example might be the expression “Jupiter is a planet,” which might be expressed using the logical statement “planet(x). x=Jupiter.”
Another such approach to proof checking systems or other similar systems is called paramodulation. Paramodulation is a method of theorem proving in which rules of inference operate on two clauses to yield a third clause. In the simplest cases, it can be a trivial statement of the transitive property, where appropriate variables are substituted for other variables; for example, if a=b and b=c, then a=c. However, the process can be used to map complex hierarchical relationships and is not confined to obvious equalities. Properties of set theory and Boolean logic can be expressed with this formalism.
No systems have yet been created that go beyond proof checking and statistical processing of language and imagery. Most importantly, no systems have been able to, by themselves, generate interesting conjectures, in a manner that could match or exceed human capabilities. Some algorithms exist that can solve problem sets created for theorem provers. Algorithms have been designed to propose interesting conjectures for solutions to riddles and puzzles, where “interesting” is defined by values such as the degree to which a particular hypothesis is “surprising,” the degree to which it is “novel,” or the degree to which it shows “complexity.” (It may be noted that this is somewhat contrary to typical principles of abductive reasoning and hypothesis drafting, which, as discussed above, emphasize simplicity and consistency. A major reason for this is that, because such systems are in their infancy, they cannot easily supplant processes that humans are good at, but can potentially supplement such systems by providing unusual reasoning that humans may not necessarily have duplicated, but which may sometimes have value.) The basic idea is to generate hypotheses from existing rules in an “ontology,” defined as a set of concepts and categories in a subject area or domain that shows their properties and the relations between them. This can be accomplished as a permutation of existing relationships, or a search through a tree like structure.
Some automation has been achieved along these lines in generating hypotheses for autonomous experimentation. For example, by using comprehensive feedback loops, it has been possible to improve the rate for serial carbon nanotube experiments by a hundred-fold. The methodology makes it possible to develop new or more accurate hypotheses, explore a large parameter space of possible experiments, generate multivariate questions, and optimize the parameters that may be applied to lab experiments. The autonomous method plans, generates, and evaluates its own experiments. However, the application of even similar types of techniques to different problems has thus far been limited, because of the need to tie the method to a rule-based format that can be optimized using these comprehensive feedback loops.